A Solution to the Binding Problem for Compositional Connectionism

Achieving compositional connectionism means finding a way to represent role-filler bindings in a connectionist system without sacrificing role-filler independence. Role-filler binding schemes based on varieties of conjunctive coding (the most common approach in the connectionist literature) fail to preserve role-filler independence. At the same time, dynamic binding of roles to fillers (e.g., by synchrony of firing) represents bindings without sacrificing independence, but is inadequate for storing bindings in long-term memory. An appropriate combination of dynamic binding (for representation in working memory) and conjunctive coding (for long-term storage and token formation) provides a platform for compositional connectionism, and has proven successful in simulating numerous aspects of human perception and cognition. Compositional Systems, Role-Filler Independence and Binding Compositional systems combine and recombine a finite number of representational elements (symbols or subsymbols) into a much larger (infinite, if recursion is permitted) number of specific structures. Examples include natural languages and formal symbol systems (e.g., propositional notation, mathematical notation, and computer programming languages). An important property of a compositional system is that the meaning and representation of individual elements in a structure do not vary as a function of their position in the structure as a whole. For example, “loves,” “John” and “Mary” can be combined to form two different propositions, loves (John, Mary) and loves (Mary, John). The meaning and representation of these elements (which can be objects, relations or relational roles) is fundamentally independent of each element’s place in the compositional structure as a whole (e.g.,, Mary is represented in the same way whether she is a lover or a beloved). At the same time, the meaning of the compositional structure does depend on the elements’ configuration—i.e., the bindings of arguments (here, John and Mary) to relational roles (lover and beloved). Copyright © 2004, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Role-Filler Independence This property of formal compositional systems is so fundamental that it is easy to overlook. Yet it is essential: If the representation of an element (role or object) varied as a function the argument or role to which it was bound, then the resulting system would not be properly described as “compositional”: Different expressions consisting of the “same” symbols would not consist of the same symbols; they would be different expressions with different symbols. They would simply be different, with nothing systematic in common. The importance of role-filler independence is apparent in human relational reasoning. Relational generalization—inferences and generalizations that depend on relations between objects rather than just the features of those objects—is only possible if relational roles are represented independently of their fillers (Hummel & Holyoak, 1997, 2003). For example, suppose someone knows that John loves Mary, Mary loves Zack, and John is jealous of Zack. This person then observes that Sally loves Tom, and Tom loves Cindy. A plausible analogical inference is that Sally will be jealous of Cindy. This inference is based on the relational correspondences (i.e., participation in corresponding roles) of John to Sally, Mary to Tom, and Zack to Cindy. Making the inference requires the reasoner to discover these correspondences and to use them to guide inferences about the second (novel) situation (e.g., it is not a plausible inference that Tom will be jealous of Sally). Importantly, it is only possible to discover and use the correspondences if the loves relation is represented in the same way regardless of who loves whom (i.e., independently of its fillers), and the people involved are represented independently of their role bindings (e.g., the same Mary is both a lover and beloved; see Hummel & Holyoak, 2003).